Task-specific effects of reward on task switching

Although cognitive control and reinforcement learning have been researched extensively over the last few decades, only recently have studies investigated their interrelationship. An important unanswered question concerns how the control system decides what task to execute and how vigorously to carry out the task once selected. Based on a recent theory of control formulated according to principles of hierarchical reinforcement learning, we asked whether rewards can affect top-down control over task performance at the level of task representation. Participants were rewarded for correctly performing only one of two tasks in a standard task-switching experiment. Reaction times and error rates were lower for the reinforced task compared to the non-reinforced task. Moreover, the switch cost in error rates for the non-reinforced task was significantly larger compared to the reinforced task, especially for trials in which the imperative stimulus afforded different responses for the two tasks, resulting in a "non-paradoxical" asymmetric switch cost. These findings suggest that reinforcement at the task level resulted in greater application of top-down control rather than in stronger stimulus-response pathways for the rewarded task

Electrophysiological indices of anterior cingulate cortex function reveal changing levels of cognitive effort and reward valuation that sustain task performance

Successful execution of goal-directed behaviors often requires the deployment of cognitive control, which is thought to require cognitive effort. Recent theories have proposed that anterior cingulate cortex (ACC) regulates control levels by weighing the reward-related benefits of control against its effort-related costs. However, given that the sensations of cognitive effort and reward valuation are available only to introspection, this hypothesis is difficult to investigate empirically. We have proposed that two electrophysiological indices of ACC function, frontal midline theta and the reward positivity (RewP), provide objective measures of these functions. To explore this issue, we recorded the electroencephalogram (EEG) from participants engaged in an extended, cognitively-demanding task. Participants performed a time estimation task for 2 h in which they received reward and error feedback according to their task performance. We observed that the amplitude of the RewP, a feedback-locked component of the event related brain potential associated with reward processing, decreased with time-on-task. Conversely, frontal midline theta power, which consists of 4-8 Hz EEG oscillations associated with cognitive effort, increased with time-on-task. We also explored how these phenomena changed over time by conducting within-participant multi-level modeling analyses. Our results suggest that extended execution of a cognitively-demanding task is characterized by an early phase in which high control levels foster rapid improvements in task performance, and a later phase in which high control levels were necessary to maintain stable task performance, perhaps counteracting waning reward valuation

Reward-based contextual learning supported by anterior cingulate cortex

The anterior cingulate cortex (ACC) is commonly associated with cognitive control and decision making, but its specific function is highly debated. To explore a recent theory that the ACC learns the reward values of task contexts (Holroyd & McClure in Psychological Review, 122, 54-83, 2015; Holroyd & Yeung in Trends in Cognitive Sciences, 16, 122-128, 2012), we recorded the event-related brain potentials (ERPs) from participants as they played a novel gambling task. The participants were first required to select from among three games in one "virtual casino," and subsequently they were required to select from among three different games in a different virtual casino; unbeknownst to them, the payoffs for the games were higher in one casino than in the other. Analysis of the reward positivity, an ERP component believed to reflect reward-related signals carried to the ACC by the midbrain dopamine system, revealed that the ACC is sensitive to differences in the reward values associated with both the casinos and the games inside the casinos, indicating that participants learned the values of the contexts in which rewards were delivered. These results highlight the importance of the ACC in learning the reward values of task contexts in order to guide action selection

Rightward-biased hemodynamic response of the parahippocampal system during virtual navigation

Phase reset of parahippocampal electrophysiological oscillations in the theta frequency range is said to contribute to item encoding and retrieval during spatial navigation. Although well-studied in non-human animals, this mechanism is poorly understood in humans. Previously we found that feedback stimuli presented in a virtual maze environment elicited a burst of theta power over right-posterior areas of the human scalp, and that the power and phase angle of these oscillations were greater following right turns compared to left turns in the maze. Here we investigated the source of this effect with functional magnetic resonance imaging. Consistent with our predictions, we found that 1) feedback encountered in the maze task activated right parahippocampal cortex (PHC), 2) right PHC was more activated by rewards following right turns compared to left turns in the maze, and 3) the rightward-biased activation was more pronounced in individuals who displayed good spatial abilities. These findings support our previous electrophysiological findings and highlight, in humans, a role for PHC theta oscillations in encoding salient information for the purpose of spatial navigation

Electrophysiological measures of conflict and reward processing are associated with decisions to engage in physical effort

Anterior cingulate cortex (ACC), a key brain region involved in cognitive control and decision making, is suggested to mediate effort- and value-based decision making, but the specific role of ACC in this process remains debated. Here we used frontal midline theta (FMT) and the reward positivity (RewP) to examine ACC function in a value-based decision making task requiring physical effort. We investigated whether (1) FMT power is sensitive to the difficulty of the decision or to selecting effortful actions, and (2) RewP is sensitive to the subjective value of reward outcomes as a function of effort investment. On each trial, participants chose to execute a low-effort or a high-effort behavior (that required squeezing a hand-dynamometer) to obtain smaller or larger rewards, respectively, while their brainwaves were recorded. We replicated prior findings that tonic FMT increased over the course of the hour-long task, which suggests increased application of control in the face of growing fatigue. RewP amplitude also increased following execution of high-effort compared to low-effort behavior, consistent with increased valuation of reward outcomes by ACC. Although neither phasic nor tonic FMT were associated with decision difficulty or effort selection per se, an exploratory analysis revealed that the interaction of phasic FMT and expected value of choice predicted effort choice. This interaction suggests that phasic FMT increases specifically under situations of decision difficulty when participants ultimately select a high-effort choice. These results point to a unique role for ACC in motivating and persisting at effortful behavior when decision conflict is high